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The present invention relates to measuring the 
performance of computer systems. 

Computer system performance, e.g. the speed 
of operation or the throughput of a system, is a 
function of the computer system hardware and the 
efficiency of the software used on the system. 
Performance can be enhanced by ensuring that the 
software is efficiently written to most effectively use 
the hardware or by modifying the hardware to 
enhance certain software function. 

The identification of performance problems re- 
quires an ability to monitor the execution of soft- 
ware on a particular hardware system and to be 
able to identify those sections of the software that 
are consuming inordinate amounts of hardware re- 
source. For example, the execution of a software 
program can be monitored to determine how much 
processing time is spent in each subroutine. 

Tracing program execution and monitoring ex- 
ecution aads significant overhead to program ex- 
ecution Thus, most software does not include 
monitor. ng lunrt-on m its basic form. Software de- 
velopers mav a.1rt instructions to the software to 
monitor selected portions, but these instructions 
are typcaJiy romovec before the final version is 
shippec to customers y placed in regular use. 

Introduction of an existing program onto new 
hardware c* porcccxion o- performance problems in 
new software may create a requirement to monitor 
software that docs not contain any inherent moni- 
toring capacity This creates a need to "instru- 
ment" tho software to measure performance. In- 
strumentation of software refers to the process of 
enabling the software to be monitored at selected 
points to capture significant system state data at 
those pom:s 

Historically, instrumentation of software was ac- 
complished ty modifying the source code for the 
software to include monitoring instructions, recom- 
piling • the source code, and then executing the 
modified software The approach has the disadvan- 
tages of requiring access to source code (which 
may not be available for commercially purchased 
software), and bomg error prone if the person 
modifies the code incorrectly. In addition, this form 
of instrumentation may introduce performance 
problems itself causing the results to be mislead- 
ing. 

A second approach to instrumentation uses 
special purpose hardware to record access to cer- 
tain computer system functions. A special monitor 
is connected to the computer to record changes in 
the physical state of the machine, e.g. when a ' 
signal is received on a certain line or when certain 
memory addresses are accessed. This approach 
has the disadvantage pf requiring the special pur- 
pose hardware. It is also limited to those functions 
that cause a recognizable physical change to the 



hardware. The approach is costly, and not generally 
applicable. ' . v^v' - 7 vfe^.^-tU * 

Yet another approach has been suggested in 
U.S. Patent Number 5,193,180 to Hastings. Has-" 
5 tings seeks to monitor memory access by expand- 
ing the program code to include specific monitoring 
instructions. Hastings avoids the need for source 
code by expanding relocatable binary files. How- 
ever, the files to be . expanded must have a full 
10 symbol table available because : of the movement of 
relative locations due to the expansion. The tech- 
nique is also not applicable to isituations where the 
symbol table has been stripped from an executable 
object to save storage space. * Finally, Hastings 
75 cannot be applied to an object already loaded into 
memory for execution due to the need to recalcu- 
late relative addresses. -" J > ■ 

Still another approach is suggested in co-pend- 
ing US Patent application " serial number 
20 07/662,521 . This method is non-rnvasive and does 
not require modifying the code being monitored. 
The system and method are implemented in a 
software program that samples instruction address- 
es to be executed by the system. Summarization 
25 of the number of times an address is referenced 
and correlation to the source code causing genera- 
tion of that instruction . provides statistics on the 
time the program spends in certain "section^ of 
code. This approach has the disadvantage of being 
30 limited to estimating time spent in code sections 
and not allowing collection of other system state 
information. It also requires the source code to be 
available to generate an assembly listing for ad- 
dress to code correlation. .^lr(^^S:^^^-\ : 
35 Monitoring of entire libraries containing a large 

number of routines has typically required the inser- 
tion of monitoring code into each of the library 
routines. This process can. be terror prone and 
leads to large expanded libraries!' In most cases 
40 the monitoring code for. a library is essentially the 
same so storage of multiple copies of redundant 
code is inefficient. " ' : . ;< --? ; - &i\wV; v £ 
A technical problem therefore exists to' provide 
a means of instrumenting a program for user de- 
45 fined performance monitoring without access to the 
program source code and without requiring ispecial 
purpose hardware monitors. A further problem ex- 
ists to provide a process for efficiently monitoring a 
shared library of routines without inserting redun- 
so dant code. / : ; a ; <i*.;' : y ' ' '. 

In accordance with the present invention* there 
. is now provided a method for monitoring a plurality 
of software programs executable on a computer 
system, each of the software programs having a 
55 plurality of computer executable instructions, the 
computer system having memory and a processor, 
the method comprising the steps of; storing a plu- 
rality of monitoring programs for monitoring soft- 
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ware execution, the plurality of monitoring pro- 
grams each being placed at an addressable loca- 
tion in the memory; storing for each of the plurality . 
of software programs a monitor translation entry for 
invoking the addressable monitoring program for 5 
the software program; and replacing one of the 
plurality of computer executable instructions in 
each of. the plurality of software programs with an 
invocation of the monitor translation entry for that 
software program. 10 

Viewing the present invention from another as- 
pect, there is now provided a system for monitoring 
execution of a software routine on a computer 
system, the system comprising: storage means for 
storing data in the computer system; means for 75 
storing a plurality of monitoring programs at ad- 
dressable locations in the storage means; means 
for associating one or more of the monitoring pro- 
grams with the software routine; and means for 
modifying the software routine to invoke the asso- 20 
dated monitoring programs. 

The present invention thus provides a system 
and method for enabling monitoring of software 
performance without requiring access to the soft- 
ware source code. 25 

The present invention also provides a system 
and method for efficiently instrumenting shared li- 
braries of routines executing on a computer sys- 
tem. Furthermore, the present invention provides a 
system and method that enables instrumenting of 30 
• shared libraries without requiring access to the 
library routine source code and, without recom- 
pilation. Still furthermore the present invention pro- 
vides a system and method for instrumenting 
shared library routines after those routines have 35 
been loaded for execution in a computer system. 

Preferred embodiments of the present inven- 
tion will now be described, by way of example 
only, with reference to the accompanying drawings, 
in which: 40 
Figure 1 is a block diagram illustrating a com- 
puter system. 

Figure 2 is a flow diagram illustrating the normal, 
flow of control to and from a library routine. 
Figure 3 is a flow diagram illustrating the flow of 45 
control in an instrumented library routine accord- 
ing to the present invention. 
Figure 4 is a flowchart depicting the steps of the 
instrumentation process according to the 
present invention. 50 
Figure 5 is a diagram illustrating the detailed 
flow of execution in a system according to the 
present invention. 

Figure 6 is a diagram illustrating the layout of a 
shared* memory segment used in the recording 55 
data in the present invention. 
A preferred embodiment of the present inven- 
tion operates on a computer system, having a pro- 



cessing unit, system memory and various in- 
put/output and other peripheral devices. The pre- 
ferred embodiment operates on and IBM RISC 
System/6000 computer running the AIX operating 
system. (IBM, RISC System/6000, and AIX are 
trademarks of the IBM Corporation.) It will be un- 
derstpod, however, that the invention can be imple- 
mented on other hardware platforms and on other 
operating systems. 

The preferred embodiment is implemented with 
a computer system having the components shown 
generally for the system 100 in Figure 1. Process- 
ing is provided by central processing unit or CPU 
102. CPU 102 acts on instruction and data stored 
' in random access memory 104. Long term storage 
is provided on one or more disks 122 operated by 
disk controller 120. A variety of other storage me- 
dia could be employed including tape, CD-ROM, or 
WORM drives. Removable storage media may also 
be provided to store data or computer process 
instructions. Operators communicate with the sys- 
tem through I/O devices controlled by I/O controller 
112. Display 114 presents data to the operator 
while keyboard 114 and pointing device 118 allow 
the operator to direct the computer. system. Com- 
munications adapter 106 controls communications 
between this processing unit and others on a net- 
work to which it connected by network interface 
108. 

Instrumentation of software leads to the moni- 
toring of a "target routine", i.e. that portion of the 
software for which data is to be collected. The 
target- routine can be a complete program, a sub- 
routine of a program, or a routine from a routine 
library. A routine library or subroutine library is a 
collection of useful software that may be used by a 
programmer to accomplish a desired function with- 
out having to specifically create source code for 
the function. For example, various mathematical 
functions such as sine, cosine, tangent, etc. can be 
collected into a library of math routines. While a 
library usually includes a number of routines, the 
term is often used to refer to even a single routine 
designed to be called by another program. A 
shared library, once loaded, is accessible to. many 
processes or users on the system. 

Each target routine has one or more entry 
points and one or more exit points. A target routine 
is invoked or called by a previous routine. The 
processor will transfer control to the target routine 
entry point. Instructions from the target routine will 
be executed until an exit back to the calling routine 
is encountered. The target routine instructions may 
include an invocation of another subroutine. In 
some cases, control will transfer to another routine 
and will never be returned to the calling routine. 
The flow of control is illustrated in Figure 2. In Fig. 
2 the "Call to Target" transfers control to the 
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instruction at address 202. Target routine instruc- 
tions 204 are executed until control is returned to 
the calling program at 206. 

Enabling routine monitoring allows system 
state information to be collected at entry to the 
target routine and at exit from the routine. Entry 
and Exit monitoring provide statistics on how much 
time is spent in any routine and an ability to 
; determine what changes to the system are caused 
by that routine. The flow of control after instrumen- 
tation according to the present invention is shown 
in Figure 3. The Call to Target still points to ad- 
dress 202. After 202, however, control is passed to 
Entry Routine 210. Entry Routine 210 collects the 
information desired by the monitor and returns 
control to the target routine. Upon Exit from the 
Target Routine, control is passed to an Exit Routine 
212 that collects additional data. 

The present invention permits the Entry and 
Exit Routines to be written in a high level language 
such as C thereby making monitoring easier for the 
average programmer. This flexibility allows the pro- 
grammer to collect precisely the information need- 
ed without introducing a great deal of complexity 
into the monitoring process. Within the Entry and 
Exit routines, the programmer can direct the sys- 
tem to send data to a printer, to a file, to the 
console, or to a shared memory segment. The 
routines also allow the function of a library member 
to be fully replaced such that newly provided code 
will be executed instead of the base code in the 
library being monitored. 

An example of a shared memory segment for 
collecting monitor events is shown in Figure 6. The 
shared memory segment 600 is an allocated area 
in the system memory that is defined as accessible 
to multiple processes. The segment preferably in- 
cludes a header area 602, a counter area 604 for 
recording counts of selected events, and a event 
area 606 for recording a number of monitor events. 
The size of the memory segment is alterable based 
on the user requirements. The size allocated deter- 
mines the total number of monitor events that can . 
be captured in event area 606. Although this 
•shared segment structure is preferred, other struc- 
tures can be used within the scope of the invention. 

The system and method of the present inven- 
tion enable software monitoring by instrumenting 
the target routines selected by the user. The sys- 
tem and method perform the necessary target rou- 
tine modifications thereby eliminating potential er- 
rors caused by incorrect manual modifications. Fig- " 
ure 4 is a flowchart depicting the steps in in- . 
strumenting a target routine or library of target 
routines according to the present invention. 

The user invokes the present invention by 
specifying the target routines and type of monitor- 
ing. This information leads to the Start of process- 



ing 400. The input is processed 402 to determine 
the actions required. Working directories for hold- 
ing modified target routines are' created 404. Next, 
instrumentation objects are inserted into the target 

s routines and libraries 406. Kernel extensions are 
loaded 408. The kernel extensions allow certain 
forms of processing to be employed. 

The insertion of instrumentation objects, step 
406 involves modifying, the software to cause ex- 

io ecution of the entry and exit routines specified by 
the user. The preferred embodiment of the present 
invention is particularly suited to instrument pro- 
grams following the C language linkage conven- 
tions. Programs written in languages other than C 

75 often follow these conventions. Instrumentation of 
the software is accomplished, in part, by changing 
portions of the executable code. The changed por- 
tions are identified by analysis of the linkage con- 
ventions. Executable software embodying linkage 

20 conventions other than the C linkage conventions 
may be instrumented using this technique. Exten- 
sion of the preferred embodiment to these other 
linkage cases can be accomplished using known 
techniques. 

25 The method of instrumenting a target routine 

depends on the type of target routine. Instrumenta- 
tion of executable programs that have not been 
stripped of their symbol table ("unstripped rou- 
tines") will be described first. Descriptions for in- 

30 strumentation of stripped routines will follow; 

The general flow of an instrumented target 
routine is shown in Figure 5. A target routine 502 is 
instrumented or encapsulated by causing the rou- 
tine to branch to the instrumentation or monitor 

35 code. 

Instrumentation of a library of routines poses 
several unique problems. The person desiring to 
monitor a library typically plans to collect, the same 
data on the execution of each routine. Prior art 

40 systems required code for collecting that data to 
be inserted into each oHhe library routines. Even 
in the commonly assigned related application in- 
strumentation of a shared library required instru- 
mentation library code to be linked with the target 

45 routine. Inclusion of the monitoring code with each 
target routine greatly expands - the size of the li- 
brary and creates a problem of ensuring that the 
correct level of monitoring code is included. Modi- 
fication of the monitoring code requires recon- 

50 structing the entire monitored library. 

The present invention solves this problem by 
introducing a demultiplexer instrumentation func- 
tion. The demultiplexer directs a target routine 
monitor request to common instrumentation code 

55 which in turn calls the appropriate entry or exit 
monitoring routine. The demultiplexer function pro- 
vides a means for inserting minimal code into the 
target routine thereby saving library space and 
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decreasing the instrumentation effort required. 

A demultiplexer entry according to the present 
invention is illustrated at 506 in Figure 5. The 
demux entry contains a load demux-entry number 
512, an instruction 514 causing a branch to com- 
mon instrumentation code 530, a number 516 re- 
presenting the displacement from the demux entry 
to the target routine, the first instruction 508 of the 
• target routine, a branch 518 to the second instruc- 
tion 505 of the target routine, a branch instruction 
520 to the user supplied exit routine 550, and a 
branch instruction '522 to the user supplied entry 
routine 552. The demux entry comprises six 
instructions and one integer _value. In the preferred 
embodiment this results in a demux entry of only 
28 bytes. The provision of one demux entry per 
library routine therefore adds very little overhead. 
Each added routine adds only 28 bytes to the 
overall size after the common code is added. 

Instrumentation of an unstripped routine prefer- 
ably links the common instrumentation code with 
the shared library. This allows the library and the 
instrumentation code to use the same Table of 
Contents or TOC and removes the requirement of 
creating a separate Table' of Contents for the in- 
strumentation code thereby saving space.' Library 
routines must first.be extracted from the library 
archive and linked with the instrumentation code. 
Once .instrumented, the library routines must be 
archived back into the library. ' 

The instrumentation routine then locates the 
first instruction of each library routine and copies it 
to the demux entry for that routine at 508. The first. • 
instruction of the routine is replaced with a branch 
instruction 504 to the instrumentation code demul- 
tiplexer (demux) entry 506. 

The flow of execution in an instrumented sys- 
tem is as follows: 

A library routine is called by another program. 
The first instruction encountered is a branch to the 
demux entry 504. Demux entry 506 contains a 
branch instruction 514 to the common instrumenta- 
tion code 530. The common instrumentation code 
530 saves the link register 532. saves the other 
registers 534, and calls 536 the user supplied entry 
routine* 550 by using the demux entry 520 to 
branch to the entry routine. Upon completion of the 
entry routine 550 processing, control returns to the 
common instrumentation code at 538.. The registers 
are restored 538. the link register is set to the 
address of a return point in the instrumentation 
code 540. A branch to the first library routine 
instruction 508 (stored in the demux entry) is ex- 
ecuted 542 and control returned to the library rou- ' 
line second instruction 505. Upon completion of 
library routine processing, control returns to the 
instrumentation routine return point 544. The regis- 
ters are saved 545 and a branch made to the exit 



routine 552 using the demux entry 522. The exit 
„ routine 552 collecis data requested on exit and 
returns control to the instrumentation code at 547. 
The registers are restored' 547, the original link 
register restored 548, and a branch is taken back 
to the address specified in the link register 549. 

User supplied routines 550, 552 can be written 
in C or other high level language and can capture 
any required system state information. An ability to 
w set and monitor a timer allows the entry/exit routine 
combination to collect data on the amount of time 
spent in each routine. 

Stripped executable modules cannot be linked 
with the instrumentation code. Instrumenting of 
75 stripped library routines therefore requires a slight- 
ly different approach. The instrumentation code 
530 is stored in a library of entry and exit routines. 
Each of the. library instrumentation routines, in turn, 
invokes a user supplied entry or exit routine for 
20 collecting data. The loader section of the library 
routine is modified to create a dependency on the 
newly created instrumentation library and to cause 
that library to be loaded prior to executing the first 
instruction of the library routine. 
25 The text section of the library routine is ex- 

panded to include a demux entry for calling the 
instrumentation library routines. The stripped rou- 
tine demux entry is larger than the demux entry for 
unstripped routines. The stripped demux entry con- 
30 tains instructions for saving, the registers and for 
calculating the addresses of the common code and 
user supplied entry and exit routines. Demux en- 
tries in the preferred embodiment contain 34 
instructions and two integer data items totalling 144 
35 bytes. Other implementations having different num- 
bers of instructions and data could be implemented 
within the scope of the invention. The data section 
of the library routine is expanded to provide stor- 
age for TOC entries of the instrumentation library 
40 making them addressable from the library routine. 
The first instruction of the library routine is copied 
to a specific offset within the inserted demux entry. 
A branch from the library routine to the entry 
section of the demux entry replaces the first in- 
45 struction of the library routine. The flow of execu- 
tion is then as described above. 

The only modification to the library routine is 
the replacement of the first instruction by a branch 
instruction. The demux entry is appended to the 
so target routine but does not expand or disrupt the 
instruction sequence of the target routine. 

Claims 

55 1. A method for monitoring a plurality of software 
programs executable on a computer system, 
each of the software programs having a plural- 
ity of computer executable instructions, the 
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2. 



computer system having memory and a pro- 
cessor, the method comprising the steps of: 

storing a plurality of monitoring programs ' 
(or monitoring software execution, the plurality 
of monitoring programs each being placed at 5 
an addressable location in the memory; 

. storing for each of the plurality of software 
programs a monitor translation entry for invok- 
ing the addressable monitoring program for the 
software program; and 10 

replacing one of the plurality of computer 
executable instructions in each of the plurality 
of software programs with an invocation of the 
monitor translation entry ior that software pro- 
gram. 15 

A method as claimed in claim 1 , comprising 
the steps of: 

executing one of the software programs; 

accessing the monitor translation entry to 20 
determine the address of the associated moni- 
toring program; 

executing the monitoring program; 

returning execution control to the software • 
program. 25 

A method as claimed in claim 1, wherein the : 
plurality of monitoring programs include com- 
mon monitoring programs common to all of the 
software programs and entry /exit monitoring 30 
programs for performing monitoring actions for 
one or more of the software programs, and., 
wherein the step of storing for each of the 
plurality of software programs a monitor trans- 
lation entry comprises the steps of: 35 

determining the common monitoring pro- 
gram address; 

receiving an indication of which of the en- 
tryexit monitor programs are associated with 
the software program; ^ 

determining the addresses of the asso- 
ciated entry/exit monitor programs; and 

storing a monitor translation entry contain- 
ing an invocation of the common monitor pro- 
gram and an address for each of the entry/exit ' 45 
•monitor programs. 



means for modifying the software routine 
to invoke the associated monitoring programs. 

5. A system as claimed in claim 4, wherein the 
means for associating the monitoring programs 
associates a first monitor program for common 
routine processing and associates a plurality of 
entry/exit monitor programs for monitor activi- 
ties, and wherein the system further com- 
prises: 

means for causing the first monitor pro- 
gram to invoke the addressable entry/exit mon- 
itor programs. 

6. A system as claimed in claim 4, further com- 
prising: 

means for collecting data generated by the 
monitoring programs. 



4. A system for monitoring execution of a soft- 
ware routine on a computer- system, the sys- 
tem comprising: so 

storage means for storing data in the com- 
puter system; 

means for storing a plurality of monitoring 
programs at addressable locations in the stor- 
age means;. 55 

means for associating one or more of the 
monitoring programs with the software routine; 
and 
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